In some coding strategies, coding of a syntax element in a frame partition (such as a macroblock, sub-block and/or block), may be conditioned or dependent on the data assigned to syntax elements from the whole neighborhood (causal and non-causal according to the coding scanning order). Typical Macroblock data coding, for example, such as that performed in the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 recommendation (hereinafter the “MPEG-4 AVC standard”) impairs the use of such coding strategies, as no data from later macroblocks in the scanning order is available.
The MPEG-4 AVC Standard proposes a slice data partitioning in three categories in which data is basically divided as follows: Category A is all data but the residual; Category B is part one of residual data; and Category C is part two of residual data. The aim of data partitioning is error resilience. At the decoder, we can merge the syntax of the three categories into one complete case and perform one pass parsing based on scanning order. The MPEG-4 AVC Standard does not allow conditional and/or dependent coding of motion data with respect to the mode data of the whole macroblock and/or block neighborhood. Also, the MPEG-4 AVC Standard does not allow the insertion of other syntax data classes within mode data and motion data.
Practical applications such as leaf merging in accordance with a first prior art approach only focus on the algorithm and do not supply an ordered scheme for proper systematic encoding of data.
The leaf merging paradigm relies on the fact that, in the field of image coding, it has recently been shown that quad-tree schemes for representing geometric image features are unable to obtain the optimal exponentially decaying rate-distortion behavior. This problem can be rectified by following rate-distortion (R-D) optimal tree pruning with a leaf merging step. In consideration of the same, the first prior art approach noted that quad-tree based motion representations for video compression suffer from the same fundamental shortcomings, which can again be overcome by leaf merging. Based on these observations, the first prior art approach proposes a non-iterative extension to the existing model used by the MPEG-4 AVC Standard, where after a first tree decomposition, neighboring blocks are adaptively merged for joint coding. This generates a series of causal and non-causal dependencies among blocks where prediction is coded, their coding mode and merging side information. For this purpose, access to some classes of data in a non-causal manner with respect to the scanning order is necessary. However, the first prior art approach does not supply a systematic and ordered way of encoding the necessary data classes for leaf merging into the coded stream. Indeed, for every block, merging hypotheses are tested. If the compression efficiency improves by merging with a neighboring block then merging is enabled for that block to be merged with the selected target block. The possible merging targets for a given block depend on the neighboring block modes and on the particular mode of the merging candidate block. Hence, the merging side information depends on all neighboring block modes and the particular mode of the merging candidate block. Finally, prediction coding will depend on the merging side information for a block which at the same time depends on the merging side information of neighboring blocks and the local and neighboring coding mode.
Turning to FIG. 1, a video encoder capable of performing video encoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral 100.
The video encoder 100 includes a frame ordering buffer 110 having an output in signal communication with a non-inverting input of a combiner 185. An output of the combiner 185 is connected in signal communication with a first input of a transformer and quantizer 125. An output of the transformer and quantizer 125 is connected in signal communication with a first input of an entropy coder 145 and a first input of an inverse transformer and inverse quantizer 150. An output of the entropy coder 145 is connected in signal communication with a first non-inverting input of a combiner 190. An output of the combiner 190 is connected in signal communication with a first input of an output buffer 135.
A first output of an encoder controller 105 is connected in signal communication with a second input of the frame ordering buffer 110, a second input of the inverse transformer and inverse quantizer 150, an input of a picture-type decision module 115, an input of a macroblock-type (MB-type) decision module 120, a second input of an intra prediction module 160, a second input of a deblocking filter 165, a first input of a motion compensator 170, a first input of a motion estimator 175, and a second input of a reference picture buffer 180.
A second output of the encoder controller 105 is connected in signal communication with a first input of a Supplemental Enhancement Information (SEI) inserter 130, a second input of the transformer and quantizer 125, a second input of the entropy coder 145, a second input of the output buffer 135, and an input of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter 140.
A first output of the picture-type decision module 115 is connected in signal communication with a third input of a frame ordering buffer 110. A second output of the picture-type decision module 115 is connected in signal communication with a second input of a macroblock-type decision module 120.
An output of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter 140 is connected in signal communication with a third non-inverting input of the combiner 190.
An output of the inverse quantizer and inverse transformer 150 is connected in signal communication with a first non-inverting input of a combiner 127. An output of the combiner 127 is connected in signal communication with a first input of the intra prediction module 160 and a first input of the deblocking filter 165. An output of the deblocking filter 165 is connected in signal communication with a first input of a reference picture buffer 180. An output of the reference picture buffer 180 is connected in signal communication with a second input of the motion estimator 175. A first output of the motion estimator 175 is connected in signal communication with a second input of the motion compensator 170. A second output of the motion estimator 175 is connected in signal communication with a third input of the entropy coder 145.
An output of the motion compensator 170 is connected in signal communication with a first input of a switch 197. An output of the intra prediction module 160 is connected in signal communication with a second input of the switch 197. An output of the macroblock-type decision module 120 is connected in signal communication with a third input of the switch 197. An output of the switch 197 is connected in signal communication with a second non-inverting input of the combiner 127.
Inputs of the frame ordering buffer 110 and the encoder controller 105 are available as input of the encoder 100, for receiving an input picture 101. Moreover, an input of the Supplemental Enhancement Information (SEI) inserter 130 is available as an input of the encoder 100, for receiving metadata. An output of the output buffer 135 is available as an output of the encoder 100, for outputting a bitstream.
Turning to FIG. 2, a video decoder capable of performing video decoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral 200.
The video decoder 200 includes an input buffer 210 having an output connected in signal communication with a first input of an entropy decoder 245. A first output of the entropy decoder 245 is connected in signal communication with a first input of an inverse transformer and inverse quantizer 250. An output of the inverse transformer and inverse quantizer 250 is connected in signal communication with a second non-inverting input of a combiner 225. An output of the combiner 225 is connected in signal communication with a second input of a deblocking filter 265 and a first input of an intra prediction module 260. A second output of the deblocking filter 265 is connected in signal communication with a first input of a reference picture buffer 280. An output of the reference picture buffer 280 is connected in signal communication with a second input of a motion compensator 270.
A second output of the entropy decoder 245 is connected in signal communication with a third input of the motion compensator 270 and a first input of the deblocking filter 265. A third output of the entropy decoder 245 is connected in signal communication with an input of a decoder controller 205. A first output of the decoder controller 205 is connected in signal communication with a second input of the entropy decoder 245. A second output of the decoder controller 205 is connected in signal communication with a second input of the inverse transformer and inverse quantizer 250. A third output of the decoder controller 205 is connected in signal communication with a third input of the deblocking filter 265. A fourth output of the decoder controller 205 is connected in signal communication with a second input of the intra prediction module 260, with a first input of the motion compensator 270, and with a second input of the reference picture buffer 280.
An output of the motion compensator 270 is connected in signal communication with a first input of a switch 297. An output of the intra prediction module 260 is connected in signal communication with a second input of the switch 297. An output of the switch 297 is connected in signal communication with a first non-inverting input of the combiner 225.
An input of the input buffer 210 is available as an input of the decoder 400, for receiving an input bitstream. A first output of the deblocking filter 265 is available as an output of the decoder 200, for outputting an output picture.